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May 8, 2998 


Subject: INVESTIGATIVE DATA WAREHOUSE 
FOlPANo. 1058805-000 


Dear Ms. Hofmann: 

The enclosed documents were reviewed under the Freedom of Information/Privacy Acts (FOlPA), Title 5, 
United States Code, Section 552/552a. Deletions have been made to protect information which is exempt from disclosure, 
with the appropriate exemptions noted on the page next to the excision. In addition, a deleted page information sheet was 
inserted in the file to indicate where pages were withheld entirely. The exemptions used to withhold information are marked 
below and explained on the enclosed Form OPCA-16a: 


Section 552 Section 552a 


□(b)(1) 

□(b)(7)(A) 

□(d)(5) 

B(b)(2) 

□(b)(7)(B) 

°G)(2) 

□ (b)(3) 

0(b)(7)(C) 

□(k)(1) 


B(b)(7)(D) 

□ (k)(2) 


S(b)(7)(E) 

□(k)(3) 


□(b)(7)(F) 

□ (k)(4) 

□ (b)(4) 

□(b)(8) 

□(k)(5) 

□(b)(5) 

□(b)(9) 

□ (k)(6) 

S(b)(6) 


□(k)(7) 


159 page(s) were reviewed and 159 page(s) are being released. 

o Document(s) were located which originated with, or contained information concerning other 
Government agency(ies) [OGA]. This information has been: 

□ referred to the OGA for review and direct response to you. 

□ referred to the OGA for consultation. The FBI will correspond with you regarding this 
information when the consultation is finished. 

B You have the right to appeal any denials in this release. Appeals should be directed in 
writing to the Director, Office of Information and Privacy, U.S. Department of Justice,1425 
New York Ave., NW, Suite 11050, Washington, D.C. 20530-0001 within sixty days from the 
date of this letter. The envelope and the letter should be clearly marked “Freedom of Information 
Appeal” or “Information Appeal.” Please cite the FOiPA number assigned to your request so that it 
may be easily identified. 

□ The enclosed material is from the main investigative file(s) in which the subject(s) of your request was 
the focus of the investigation. Our search located additional references, in files relating to other 
individuals, or matters, which may or may not be about your subject(s). Our experience has shown, 
when ident, references usually contain information similar to the information processed in the main file(s). 
Because of our significant backlog, we have given priority to processing only the main investigative file(s). 



If you want the references, you must submit a separate request for them in writing, and they will be 
reviewed at a later date, as time and resources permit. 

13 See additional information which follows. 


Sincerely yours. 


David M. Hardy 
Section Chief 
Record/Information 
Dissemination Section 
Records Management Division 


Enclosure(s) 


vn, ir documents represent the fifth of a series of interim releases that will be made with regard to 

your request pertaining to the Investigative Data Warehouse. ^ 

VI pages contain information about other subject matters We 

consider this information Outside the Scope” of your request, and have redacted the information and marked it as 
uutsiQ6 the ocope. 

artiiaiiw flQ«o page of this correspondence reflects that 159 pages were reviewed for this release. There were 

to thfreqSt^ ^ reviewed, however a further review determined that 8823 pages of this material are not responsive 



EXPLANATION OF EXEMPTIONS 


(b)(1) 

(b)(2) 

(b)(3) 

(b)(4) 

(b)(5) 

(b)(6) 

(b)(7) 


(b)(8) 

(b)(9) 

(d)(5) 

G)(2) 

(k)(l) 

(k)(2) 

(k)(3) 

(k)(4) 

(k)(5) 

(k)(6) 

(k)(7) 


SUBSECTIONS OF TITLE 5, UNITED STATES CODE, SECTION 552 

(A) specifically authorized under criteria established by an Executive order to be kept secret in the interest of national defense 
policy and (B) are in fact properly classified to such Executive order; 

related solely to the internal personnel rules and practices of an agency; 


or foreign 


specifically exempted from disclosure by statute (other than section 552b of this title), provided that such statute(A) requires that the 
matters be withheld from the public in such a manner as to leave no discretion on issue, or (B) establishes particular criteria for 
withholding or refers to particular types of matters to be withheld; 


trade secrets and commercial or financial information obtained from a person and privileged or confidential; 

inter-agency or intra-agency memorandums or letters which would not be available by law to a party other than an agency in litigation 
With the agency; o ^ t> 


personnel and medical files and similar files the disclosure of which would constitute a clearly unwarranted invasion of personal privacy; 

records or information compiled for law enforcement purposes, but only to the extent that the production of such law enforcement 
records or information ( A ) could be reasonably be expected to interfere with enforcement proceedings, ( B ) would deprive a person 
of a right to a fair trial or an impartial adjudication, ( C ) could be reasonably expected to constitute an unwarranted invasion of personal 
privacy, ( D ) could reasonably be expected to disclose the identity of confidential source, including a State, local, or foreign agency or 
authority or my private institution which furnished information on a confidential basis, and, in the case of record or information compiled 
by a criminal law enforcement authority m the course of a criminal investigation, or by an agency conducting a lawful national security 
intelligence investigation, information furnished by a confidential source, ( E ) would disclose techniques and procedures for law 
enforcement investigations or prosecutions, or would disclose guidelines for law enforcement investigations or prosecutions if such 
disclosure could reasonably be expected to risk circumvention of the law, or ( F ) could reasonably be expected to endanger the life or 
physical safety of any individual; ^ 


contained in or related to examination, operating, or condition reports prepared by, on behalf of, or for the use of an agency responsible for 
the regulation or supervision of financial institutions; or s> j f 

geological and geophysical information and data, including maps, concerning wells. 

SUBSECTIONS OF TITLE 5, UNITED STATES CODE, SECTION 552a 

information compiled in reasonable anticipation of a civil action proceeding; 

material reporting investigative efforts pertaining to the enforcement of criminal law including efforts to prevent, control or reduce 
crime or apprehend criminals; 


information which is currently and properly classified pursuant to an Executive order in the interest of the national defense or foreign 
policy, for example, information involving intelligence sources or methods; 


investigatory material compiled for law enforcement purposes, other than criminal, which did not result in loss of a right benefit or 

privilege undj Federal programs, or which would identify a source who furnished information pursuant to a promise that his/her identity 
would be held m confidence; ^ 


material maintained in connection with providing protective services to the President of the United States or any other individual pursuant 
to the authority of Title 18, United States Code, Section 3056; 


required by statute to be maintained and used solely as statistical records; 


investigatory material compiled solely for the purpose of determining suitability, eligibility, or qualifications for Federal civilian 
employment or for access to classified information, the disclosure of which would reveal the identity of the person who furnished 
information pursuant to a promise that his/her identity would be held in confidence; 


testing or examination material used to determine individual qualifications for appointment or promotion in Federal 
release of which would compromise the testing or examination process; 


Government service the 


material used to determine potential for promotion in the armed services, the disclosure of which would reveal the identity of the person 
who furnished the material pursuant to a promise that his/her identity would be held in confidence. 


FBI/DOJ 
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FBI Automated 
Case System 
(ACS) 

Secret and below 


InteiPIus 

Filerooms 

FBI InteiPIus 

Secret and below 


SAMNet 

FBI Secure 

Automated 

Messaging 

Network 

(SAMNet) 

Secret and below 


VGTOF 

FBI National 
Crime Information 
Center (NCIC) 

SBU 


JICI 
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Management 
Division (RMD), 
Document 
Laboratory 
(DocLab), FBIHQ 

Secret and below 


Open 

Source 

News 

MiTAP 

Unclassified 
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Security Concept of Operations (S-CONOPS) 
Investigative Data Warehouse (IDW) Program 

Document Version 1.2 
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Outside the Scope 


The table below lists the data and sources for the Operational Subsystem. 


ijigsias 




ACS 

Electronic 
Case Files 
(ECF) 

FBI Automated 
Case Support 
(ACS) 

Secret and below 


IntelPlus 

Filerooms 

FBI IntelPIus 

Secret and below 


SAMNet 

FBI Secure 

Automated 

Messaging 

Network 

(SAMNet) 

Secret and below 


VGTOF 

FBI National 
Crime Information 
Center (NCIC) 

SBU 


JICI 

FBI Records 
Management 
Division (RMD), 
Document 
Laboratory 
(DocLab), FBIHQ 

Secret and below 


Open 

Source 

News 

MiTAP 

Unclassified 



Operational Subsystem Data Sources 


The data flow begins at the external data sources. These sour ce.s include various FBI 
databases including ACS, SAMNET, Intelplus and several others./ 
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■Outside the Scope 


2.3.2 Special Projects Team (SPT) Subsystem 

The Special Projects Team (SPT) Subsystem allows for the rapid import of new specialized 
data sources. These data sources are not made available to the general IDW users but instead are 
provided to a small group of users who have a demonstrated “need-to-know”. The SPT System 
is similar in function to the IDW-S system, with the main difference is a different set of data 
sources. The SPT System allows it’s users to access not only the standard IDW Data Store but 
the specialized SPT Data Store. 

The table below lists the data and sources for the SPT Subsystem. 
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FBI Case 
Data 

FBI Automated 
Case System 
(ACS) 

Secret and below 

Financial 

Center 

(FinCen) 

Data 


SBU 

Various 
Lists of 
Data 

Various Sources, 
e.g., 

Transportation 

Security 

Administration 

(TSA) 

SBU 

Various 

CTD 

Datasets 

FBI 

Counterterrorism 
Division (CTD) 

Secret and below 

Telephone 

Data 

Database of 
telephone 
numbers from 
ACS indexed by 
phone number, 
names, addresses, 
and case numbers 

Secret and below 

SPTData^ 

m 


Outside the Scope 
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Open 

Source 

News 


FBI Automated 
Case System 
(ACS) 


FBI Secure 

Automated 

Messaging 

Network 

(SAMNet) 



FBI Records 
Management 
Division (RMD), 
Document 
Laboratory 
(DocLab), FBIHQ 


MiTAP 


Secret and below 
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IDW-S VI. 0 Data Migration Plan 
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SCOPE V2. 0/ID W VI. 0 Data Sets 

ACS ECF(CT Case Files): 

^ ASCII flat files (metadata document text) and WordPerfect (document) binaries 


♦ Data is synchronized once a day, which includes processing of adds, deletes, and modifications 
Intelplus (CT Filerooms): 

♦ Scanned TIFF images and OCR’ed ASCH text documents 


♦ Data is synchronized once a week, which includes processing of adds, deletes, and modifications 
SAMnet: 

♦ ASCII files in standar d cable traffic message format (all cap itals with specific header) 

♦ L I 

♦ SAMNET data is provided/updated three times a day 

JICI: 

♦ Scanned TIFF images and QCR’ed ASCE text (pages) 


♦ Static file system ^ ' 

VGTOF: 

♦ ^scn flat files (dat a/metadata, one row per subject) and JPEG im age binaries (none, one, or multiple per subject) 

♦ At present, updated file being provided in its entirety on a weekly basis 

Open Source: 

♦ The Open Source Library is collected fi-om global news sources that are either in English or have been translated 
mto Enghsh 


♦ The Open Source Library is updated once a day 
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Version: 0.6 

ALL INFORKATIOK CONTAINED 
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Investigative Data Warehouse (IDW) - Secret 
System Security Plan 
3_December 2003 
— Version 0.6 


Investigative Data Warehouse flDW) - 
System Security Plan 

INTRODTJCTTON 

%e Federal Bur^u of Investigation’s (FBI) Investigative Data Warehouse (IDW-S) is an 
initial data warehouse, content management and data mining system that will permit FBI 
investigative, anal^ical. administrative and intelligence personnel to access aggregated 
ate previously only available through individual applications. The IDW-S system will be 

Et^eTn^Jn^^^^ Counter-Te^orism/Collaboration Operational 



Outside the Scope 


Date processed by the system will include the following data sets: 


• Approved case files from the FBI’s Automated Case Support (ACS) case 

management system; ^ 

* of die Joint Intelligence Committee Investigation 
(.JlCl) defined archived documents; 
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Investigative Data Warehouse (IDW) - Secret 
System Security Plan 
. . SJDecember 2003 

Version 0.6 

• Secure Automated Messaging Network (SAMNet) message traffic; 

• IntelPIus File Rooms; and 

• Violent Gang and Terrorist Organization File (VGTOF”) from the 
Criminal Justice Information Systems (CJIS) Division; 


• Defense Advanced Research Projects Agency (DARPA) Translingual 
Information Detection, Extraction and Summarization (TIDES) Open 
Source Data 



- 2 - 
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Investigative Data Warehouse (DDW) - Secret 
System Security Plan 
3_December 2003 
Version 0.6 


Data Sources 

roW-S contains data from the following data sources with security classification indicated 
in brackets: 

• Automated Case Support (ACS) System, Electronic Case File (ECF) 
Subsystem [Secret and below] 

• IntelPlus [Secret and below] 

• Secure Automated Message Network - Secret (SAMNet-S) [Secret and 
below] 

• NCIC Violent Gang and Terrorist Offender File (VGTOF) [Sensitive But 
Unclassified] 

• Joint Intelligence Committee Investigation (JICI) [Secret and below] 

• DARPA/nDES Open Source News [Unclassified] 

Data from each source is held in an identified source Library against which users can direct 
the analytical tools. 

Data Ingest 

IDW-S receives data from sources by FTP over FBINet 
SAMNet) and CD-ROM A^GTOF. Open Source News)j 




'H Deleted: SAMNet 


Data is received by the Ingest Subsystem 


- Received data is parsed into individual serial records (in the case of ACS 
ECF), documents (IntelPlus, JICI, Open Source News), messages 
(SAMNet), or files (VGTOF). 



Outside the Scope 
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Investigative Data Warehouse (IDW) - Secret 
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Outside the Scope 


3.5. Data Processed 


IDW-S processes the following data: 

3.5.1 ACSECF 

IDW-S contains a subset of the ECF (Electronic Case File) subsystem of the 
Automated Case System (ACS). This subset consists of serials in those case 
classifications/sub classifications that have been officially sanctioned for inclusion 
in now. For each such serial, the ECF data includes metadata and text. IDW-S is 
synchronized against the ECF system once a day, a process which consists of 
receiving and processing the previous day’s increment of ADD. MOD. and 
DELETE records.] ^ ^ 


3.5.2 IntelFlus 


Outside the Scope 


IDW-S currently contains several IntelPlus counter-terrorism (CT) Filerooms: 


b2 

b7E 
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3.5.3 SAMNet 


b2 

b7E 


b2 

b7E 


The Secure Automated Messaging Network fSAMNet) data source consists of cable 
traffic messages received by the FBIJ 

J SAMNet provides only ADD record 

types, and SAMNet data is updated in IDW-S three times a day. SAMNet data is 


3.5.4 VGTOF ' b2 

Id V 

Violent Gang Terrorist Offender File (VGTOF) data is provided by the FBI 
National Crime Information Center (NCIC). VGTOF data includes two 
components: Data/metadata for each named individual/offender and potentially 
multiple JPEG images per individual.l 1 
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3.5.5 JICI 


The Joint Intelligence Committee Investigation (JICI) data collection was created following 
the September 1 1 attacks. Paper counter-terrorism files/folders from all FBI Field Offices 




paateaj 


s 


I JICI is a static collection that 

1 


3.5.6 Open Source News 


The IDW-S VI. 0 system contains an Open Source News library collected by the 
DARPA TIDES Program. These are primarily news source from around the world 
that are either in English or have been translated into English. Op en Source News 
data is r^ eived in the form of text files (one per news article) and | 

I I r he Open Source News data goes into IDW-S once a day. Open Source 

News daXi 

Open Source News material is derived from the following sources: 

Addis Ababa Tribune - http://www.addistribune.com/ 

Agenda Brasilia - http://www.radiobras.gov.br/ 

Al-Ahram (Egypt, weekly English version) - http://weekly.ahram.org.eg/ 

AllAfrica.com - http://allafHca.com/ 

Arabic News - http://www.arabicnews.com/ansub/ 

Asahi Shimbun - http://www.asahi.com/english/ 

Asia Times (Hong Kong) - http://www.atimes.com/ 

Bangkok Post - http://www.bangkokpost.nef/ 

.Christian Science Monitor - http://www.csmonitor.com/ 

Crescent International - http://www.muslimedla.com/mainpage.htm 
Daily Telegraph (London, England) - http://news.telegraph.co.uk/ 

Dawn (Karachi, Pakistan) - http://www.dawn.com/ 

Debka (Israel) - http://www.debka.com 

East Africa Daily Nation - www.nationaudio.com/News/DailyNation/Today/ 

Gulf News (UAE) - http://www.gulf-news.com/ 

Ha'aretz (Israel) - http://www.haaretzdaily.com/ 

I IFRC international Fedgi^^^^ - ht^://www.ifrc.org/ 

IRIN Integrated Regional Information Network - littpi/Avww.irinnews.or^ 

Iraq Press News Agency - http://www.iraqpress.org/ 


b2 

b7E 


b2 

b7 


Deleted; Iniemation 
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Islamic Republic News Agency (Iran) - http://www.ima.com/en 
Jakarta Post - http://www.thejakartapost.com/ 

Janes - http://www.janes.com/ 

Jordan Times - http://www.jordantimes.com/ 

L Osservatore Romano - www.vatican.va/news_services/or/or_eng/text.html 
Lagos (Nigeria) Guardian - http://www.guardiannewsngr.com/ 

Lahore (Pakistan) Nation - http://www.nation.com.pk/ 

Lebanon Daily Star (Beirut) - httpV/www.dailystar.com.lb/ 

Malaysian Star - http://thestar.com.my/ 

Manila Bulletin - http://www.mb.com.ph/ 

Manila Times - http://www.manilatimes.net/ 

Miami Herald - http://www.miami.com/ 

Moscow Times - http://www.themoscovrtimes.com/ 

National Post and CP - http://www.canada.com/ 

New Straits Times (Kuala Lumpur) - http://www.emedia.com.my/Current_News/NST/ 

PETRA (Jordanian News Agency) - http://www.petra.gov.jo 
Pakistan Observer (Islamabad) - http://pakobserver.net/ 

Palestine Chronicle - http://palestinechronicle.com/ 

People's Daily (China) - http://english.peopledaily.com.cn/ 

Philippine Star - http://vvww.philstar.com/philstar/ 

Pravda - http://english.pravda.ru/ 

ProMed - epidemiology mailing list 

Russian Information Agency Novosti - http://en.rian.ru/ 

Russian Issues (Misc. Russian news) - http://www.therussianissues.com/ 

Russian Observer - http://www.russianobserver.com/ 

SABA (The News Agency of Yemen) - http://www.sabanews.gov.ye 
Saudi Gazette - http://www.saudigazette.com.sa/sgazette/ 

South African Dispatch - http://www.dispatch.co.za/ 

Sydney Morning Herald - http://www.smh.com.au 
Tehran Times - http://www.tehrantimes.com/ 

Times of India - http://timesofindia.indiatimes.com/cms.dll 
UNHCR UN High Commissioner on Refugees - http://www.unhcr.ch/ 

Ummah News - http://www.ummahnews.com/ 

Uzbekistan Report - http://www.uzreport.com/eng/ 

Washington Post - http://www.washingtonpost.com/ 

XinHua News Service - http://www.xinhuanet.com/english/ 

Yemen Times (weekly) - http://www.yementimes.com/ 

Yomiuri Shimbun (Japan) - http://www.yomiuri.co.jp/ 
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3.5.7 Summary of Data Sources 




isaiiiB*® 

ACS 

FBI Automated Case 
System (ACS) 

FTP 

IntelPlus 

FBI Intel Plus 

FTP 

SAMNet 

FBI Secure Automated 
Messaging Network 
(SAMNet) 

FTP 

VGTOF 

FBI National Crime 
Information Center tNCIO 

CD-ROM 

JICI 

FBI Records Management 
Division (RMD), 
Ooeument Laboratory 
(DocLab), FBIHQ 

Multiple 

Open 

Source 

News 

Translingual Information 
Detection, Extraction and 
Summarization (TIDES) 
•’rogram. Defense Advanced 
Research Projects Agency 
(DARPA) 

CD-ROM ' 
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3.9.9 Indirect Connections 

IDW-S connects lo the followina soui’ces of dala: 


Outside the Scope 






1 

FRI AuUinialud 

Sccrel 

FBI 

FTP 


Case .System (ACS') 


l-UI Intel Plus 

■Secret 

FBI 

FTP 
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INVESTIGATIVE DATA WARF.HOTTSF. 




- SECRET 


SYSTEM SECTJRTTY PLAN 


INTRODUCTION 

The Federal Bureau of Investigation’s (FBI) Investigative Data Warehouse (IDW-S) is an initial 
data warehouse, content management and data mining system that will permit FBI investigative, 
anal 3 dical, admimstrative and intelligence personnel to access aggregated data previously only 
available through individual applications. The IDW-S system will be authorized to process 
classified national security data up to, and including, Secret. The IDW-S system is the successor 
of the Secure Counter-Terrorism/Collaboration Operational Prototype F.nvironmRnt (SCOPE”). 



Data processed by the system will include the following data sets: outside the s cope 

• Approved case files from the FBI’s Automated Case Support (ACS) case management 
system; 

• Electronic versions of the Joint Intelligence Committee Investigation (JICI) defined 
archived documents; 

• Secure Automated Messaging Network (SAMNet) message traffic; 

• IntelPlus File Rooms; and 
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• Violent Gang and Terrorist Organization File (VGTOF”) from the Criminal Justice 
Information Systems (CJIS) Division; 

• Defense Advanced Research Projects Agency (DARPA) Translingual Information 

Detection, Extraction and Summarization (TIDBSj Open Source Data 


Outside the Scope 
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Investigative Data Warehouse (IDW) - Secret 
System Security Plan 

INTRODUCTION 

The Federal Bureau of Investigation’s (FBI) Investigative Data Warehouse (IDW-S) is an initial 
data warehouse, content management and data mining system that will permit FBI investigative, 
analytical, administrative and intelligence personnel to access aggregated data previously only 
available through individual applications. The IDW-S system vrill be authorized to process 
classified national security data up to, and including, Secret The EDW-S system is the successor 
of the Secure Counter-Terrorism/Collaboration Operational Prototype Environment (SCOPE”). 


Outside the Scope 


Data processed by the system will include the following data sets: 

• Approved case files from the FBI’s Automated Case Support (ACS) case 
management system; 

• Electronic versions of the Joint Intelligence Committee Investigation (JICI) 
defined archived documents; 

• Secure Automated Messaging Network (SAMNet) message traffic; 
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• IntelPIus File Rooms; and 

• Violent Gang and Terrorist Organization File (VGTOF”) from the Criminal 
Justice Information Systems (CJIS) Division; 

• Defense Advanced Research Projects Agency (DARPA) Translingual 
Information Detection, Extraction and Summarization (TIDES) Open Source 
Data 


Outside the Scope 
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• Automated Case Support (ACS) System, Electronic Case File (ECF) Subsystem 
[Secret and below] 

• IntelPlus [Secret and below] 

• Secure Automated Message Network - Secret (SAMNet-S) [Secret and below] 

• NCIC Violent Gang and Terrorist Offender File (VGTOF) [Sensitive But 
Unclassified] 

• Joint Intelligence Committee Investigation (JICI) [Secret and below] 

• DARP A/TIDES Open Source News [Unclassified] 

Data from each source is held in an identified source Library against which users can direct the 
analytical tools. 

Data Ingest 

IDW-S receives data from sources by FTP over FBINet (ACS ECF, IntelPlus, and SAMNetl and 
CD-ROM (VGTOF. Open Source News'! 


•[ 



Data is received by the Ingest Subsystem 


I Outside the Scope 

Received data is parsed into individual serial records (in the case of ACS 
ECF), documents (IntelPlus, JICI, Open Source News), messages (SAMNet) 
or files (VGTOF). 
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have been officially sanctioned for inclusion in IDW. For each such serial, the ECF data includes 
metadata and ten. IDW-S is synchronized against the ECF system once a day, a process which 
consists of receiving and processing the previous day’s increment of ADD, MOD, and DELETE 


3.5.2 IntelPIus 


Outside 


IDW-S curre ntly contains several In telPIus counter-terrorism (CT) Filerooms: 




3.5.3 SAMNet 


The Secure Automated Messaging Network (SAMNet) data source consists of cable traffic 
messages received by the FBlf n 

1 iiAMNet provides only ADD record types, and SAMNet data is updated in IDW-S three 

times a day. SAMNet data is| ‘ 


- 22 - 


the Scope 


b2 

b7E 


b2 

b7E 


UNCLASSIFIED // FOR OFFICIAL USE ONLY 



UNCLASSIFIED // FOR OFFICIAL USE ONLY 


Investigative Data Warehouse (IDW) - Secret 
System Security Plan 
5_Decemfaer 2003 
Version 0.7 


3.5.4 VGTOF 


Violent Gang Terrorist Offender File (VGTOF) data is provided by the FBI National Crime 
Information Center (NCIC). VGTOF data includes two components: Data/metadateforea^ 
_JtamedJndividual/offender^and^potentialIyjmulti^ 


3.5.5 JICI 


The Joint Intelligence Committee Investigation (JICI) data collection was created following the 
September 1 1 attacks. Paper counter-terrorism files/folders from all FBI Field Office j 


/ JICI is a static collection that is not updated!! 


3.5.6 Open Source News 

The IDW-S Vl.O system contains an Open Source News library collected by the DARPA TIDES 
Program. These are primarily news source from around the world that are either in English or 
have been translated into English. Onen Source News data is received in the form of text files 

(one per news article) anq IThe Open Source News data goes into I DW-S 

once a day. Open Source News dat ^ | 

Open Source News material is derived from the following sources: 

Addis Ababa Tribune - http://www.addistribune.com/ 

Agenda Brasilia - http://www.radiobras.gov.br/ 

Al-Ahram (Egypt, weekly English version) - http://weekly.ahram.org.eg/ 

AllAfrica.com - http://allafrica.com/ 

Arabic News - http://www.arabicnews.com/ansub/ 

Asahi Shimbun - http://www.asahi.com/english/ 

Asia Times (Hong Kong) - http://www.atimes.com/ 

Bangkok Post - http://www.bangkokpost.net/ 

Christian Science Monitor - http://www.csmonitor.com/ 
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Crescent International - http:/Avww.muslimedia.com/mainpage.htm 
Daily Telegraph (London, England) - http://news.telegraph.co.uk/ 

Dawn (Karachi, Pakistan) - http://www.dawn.com/ 

Debka (Israel) - http://www.debka.com 

East Africa Daily Nation - www.nationaudio.com/News/DailyNation/Today/ 

Gulf News (UAE) - http://www.gulf-news.com/ 

Ha'aretz (Israel) - http://www.haaretzdaily.com/ 

IFRC International Federation of the Red Cross - http://www.iffc.org/ 

IRIN Integrated Regional Information Network - http://www.irinnews.org/ 

Iraq Press News Agency - http://www.iraqpresS.org/ 

Islamic Republic News Agency (Iran) - http://www.ima.com/en 
Jakarta Post - http://www.thejakartapost.com/ 

Janes - http://www.Janes.com/ 

Jordan Times - http://www.jordantimes.com/ 

L'Osservatore Romano - www.vatican.va/news_services/or/or_eng/text.htm! 

Lagos (Nigeria) Guardian - http://www.guardiannewsngr.com/ 

Lahore (Pakistan) Nation - http://www.nation.com.pk/ 

Lebanon Daily Star (Beirut) - http://www.dailystar.com.lb/ 

Malaysian Star - http://thestar.com.my/ 

Manila Bulletin - http://www.mb.com.ph/ 

Manila Times - http://www.manilatimes.net/ 

Miami Herald - http://www.miami.com/ 

Moscow Times - http://www.themoscowtimes.com/ 

National Post and CP - http://www.canada.com/ 

New Straits Times (Kuala Lumpur) - http://vww.emedia.com.my/Current_News/NST/ 

PETRA (Jordanian News Agency) - http://www.petra.gov.jo 

Pakistan Observer (Islamabad) - http://pakobserver.net/ 

Palestine Chronicle - http://palestinechronicle.com/ 

People's Daily (China) - http://english.peopledaily.com.cn/ 

Philippine Star - http://www.philstar.com/philstar/ 

Pravda - http://english.pravda.ru/ 

ProMed - epidemiology mailing list 

Russian Information Agency Novosti - http://en.rian.ru/ 

Russian Issues (Misc. Russian news) - http://www.therussianissues.com/ 

Russian Observer - http://www.russianobserver.com/ 

SABA (The News Agency of Yemen) - http://www.sabanews.gov.ye 

Saudi Gazette - http://www.saudigazette.com.sa/sgazette/ 

South African Dispatch - http://www.dispatch.co.za/ 

Sydney Morning Herald - http://www.smh.com.au 

Tehran Times - http://www.tehrantimes.com/ 

Times of India - http://timesofindia.indiatimes.com/cms.dll 

UNHCR UN High Commissioner on R efugees - http://wwrw.unhcr.ch/ 

— 
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Ummah News - http://www.ummahnews.com/ 

Uzbekistan Report - ht^://www.uzreport.com/eng/ 
Washington Post - http://www.washingtonpost.com/ 
XinHuaNews Service - http.V/www.xinhuaneLcom/english/ 
Yemen Times (weekly) - http://www.yementimes.com/ 
Yomiuri Shimbun (Japan) - http://www.yomiuri.co.jp/ 

3.5.7 Summary of Data Sources 
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3.9.9 Indirect Connections 

IDW-S connects to the following sources of data: 
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Case System 
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FBI Intel Plus I Secret 


UNCLASSIFIED // FOR OFFICIAL USE ONLY 



UNCLASSIFIED // FOR OFFICIAL USE ONLY 


Investigative Data Warehouse (BDW) - Secret 
System Security Plan 
5_December 2003 
Version 0.7 


SSSfiOg 

& 

msmsmssmm 

m 



FBI Secure 

Automated 

Messaging 

Network 

(SAMNet) 

Secret 
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Unclassified/SBU 
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Investigative Data Warehouse 


System Security Plan 


- Secret 


INTRODUCTION 

The Federal Bureau of Investigation’s (FBI) Investigative Data Warehouse Secret System (IDW- 
S) is an initial data warehouse, content management and data mining system that will permit FBI 
investigative, analytical, administrative and intelligence personnel to access aggregated data 
previously only available through individual applications. The IDW-S system will be authorized 
to process classified national security data up to, and including, Secret. The IDW-S system is the 
successor to the Secure Collaboration Operational Prototype Environment (SCOPE), which 
originally was named the Secure Counter-Terrorism Operational Prototype Environment. 



Data processed by the system will include the following data sets: 


• Approved case files fi-om the FBI’s Automated Case Support (ACS) case 
management system; 


Outside the Scope 


• Electronic versions of the Joint Intelligence Committee Investigation (JICI) 
defined archived documents; 
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• Secure Automated Messaging Network (SAMNet) message traffic; 

• IntelPlus File Rooms; and 

• Violent Gang and Terrorist Organization File (VGTOF”) from the Criminal 
Justice Information Systems (CJIS) Division; 


• Defense Advanced Research Projects Agency (DARPA) Translingual 
Information Detection, Extraction and Summarization (TIDES) Open Source 
Data 



Outside the Scope 
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3.5. Data Processed 


IDW-S processes data from the following data sources, with the security classification of the 
source indicated in brackets: 

• Automated Case Support fACSI System. Electronic Case File lECFI Subsystem — 
rSecret and belowl 

• IntelPlus rSecret and belowl 

• Secure Automated Message Network — Secret (SAMNet-S1 fSecret and belowl Outside the Scope ' 

• NCIC Violent Gang and Terrorist Offender File CV GTOF) ISensitive But 
Unclassifiedi 

• Joint Intelligence Committee Investigation IJICD fSecret and belowl 

• DARPA/TTDES Open Source News FUnclassifiedl 


DATASET 

NAME 


DATA SOURCE 


TRANSFER 

METnOD 


ASS 

Electronic 
Case Files 
(ECFI 


FBI Automated 
Case System 
CACSl 


FTP 


IntelPlus 

Filerooms 


FBI IntelPlus 


FTP 
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b2 
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Outside the Scope 


3.5.1 ACSECF 

IDW-S contains a subset of the ECF (Electronic Case File) subsystem of the Automated Case 
System (ACS). This subset consists of serials in those case classifications/sub classifications that 
have been officially sanctioned for inclusion in IDW-S. For each such serial, the ECF data 


includes metadata 
which consists of 
DELETE records. 

and text. IDW-S is synchronized against the ECF system once a day, a process 

|-eceiving and orocessine the nrevious dav’s increment of ADD, MOD. and 



3.5.2 IntelPlus -Outside the Scope 

IDW-S currently contains several IntelPlus counter-teirorism (CT) Filerooms: 


b2 

b7E 
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3.5.3 SAMNet 


h2 
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The Secure Automated Messa ging Netvyork (SAMNet) data source consists of cable traffic 
messages received by the FBI.| 


\SAMNet provides only Abt) record types, and SAMNet data is undated in TD W-S three 


times a day. SAMNetl 


3.5.4 VGTOF 


bZ 

hlE 


Violent Gang Terrorist Offender File (VGTOF) data is provided by the FBI National Crime 
Information Center (NCIC). VGTOF data includes two components: Data/metadata for each 

named individual/offender and p otentially multiple JPEG images per individual. The 

data/metadata for each individua/ 


3.5.5 JICI 


hZ 

b7E 


. The Joint Intelligence Committee Investigation (JICI) data collection was created fol lnwinpr the 
September 11 attacks. Paper counter-terrorism files/folders from all FBI Field Office^ 1 b2 

I 1 b7E 
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I JICI is a static collection that is not updatedL 1 

3.5.6 Open Source News 

The BDW-S VI. 0 system contains an Open Source News library collected by the DARPA TIDES 
Program. These are primarily news source from around the world that are either in English or 
have been translated into English. Open Source News data is received In the form of text files 
(one per news articl j I The Open Source News data goes into IDW-S 

once a day. Open Source News date ] 




Outside the Scope 
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Page 26: [1] Deleted Neil Gerr 

Open Source News material is derived from the following sources: 

Addis Ababa Tribune - http://www.addistribime.com/ 

Agencia Brasilia - http://www.radiobras.gov.br/ 

AI-Ahram (Egypt, weekly English version) - http://weekly.ahram.org.eg/ 
AllAfrica.com - ht^://allafrica.com/ 

Arabic News - http://www.arabicnews.com/ansub/ 

Asahi Shimbun - http://www.asahi.com/engiish/ 

Asia Times (Hong Kong) - http://www.atimes.com/ 

Bangkok Post - http://www.bangkokpost.net/ 

Christian Science Monitor - http://www.csmonitor.com/ 

Crescent International - http://www.muslimedia.com/mainpage.htm 
Daily Telegraph (London, England) - http://news.telegraph.co.uk/ 

Dawn (Karachi, Pakistan) - http://www.dawn.com/ 

Debka (Israel) - http://www.debka.com 

East Africa Daily Nation - www.nationaudio.com/News/DailyNation/Today/ 

Gulf News (UAE) - http://www.gulf-news.com/ 

Ha'aretz (Israel) - http://www.haaretzdaily.com/ 

IFRC International Federation of the Red Cross - http://www.ifrc.org/ 

IRIN Integrated Regional Information Network - http://www.irinnews.org/ 

Iraq Press News Agency - http://www.iraqpress.org/ 

Islamic Republic News Agency (Iran) - http://www.ima.com/en 
Jakarta Post - http://www.thejakartapost.com/ 

Janes - http://www.janes.com/ 

Jordan Times - http://www.jordantimes.com/ 

L'Osservatore Romano - wvw. Vatican. va/news_services/or/or_eng/text.html 
Lagos (Nigeria) Guardian - http://www.guardiannewsngr.com/ 

Lahore (Pakistan) Nation - http://www.nation.com.pk/ 

Lebanon Daily Star (Beirut) - http://www.dailystar.com.lb/ 

Malaysian Star - http://thestar.com.my/ 

Manila Bulletin - http://www.mb.com.ph/ 

Manila Times - http://www.manilatimes.net/ 

Miami Herald - http://www.miami.com/ 

Moscow Times - http://www.themoscowtimes.com/ 

National Post and CP - http://www.canada.com/ 

New Straits Times (Kuala Lumpur) - http://www.emedia.com.my/Current_News/NST/ 
PETRA (Jordanian News Agency) - http://www.petra.gov.jo 
Pakistan Observer (Islamabad) - http://pakobserver.net/ 

Palestine Chronicle - http://palestinechronicle.com/ 

People's Daily (China) - http://english.peopledaily.com.cn/ 

Philippine Star - ht^://www.philstar.com/philstar/ 

Pravda - http://english.pravda.ru/ 

ProMed - epidemiology mailing list 

Russian Information Agency Novosti - http://en.rian.ru/ 

Russian Issues (Misc. Russian news) - http://www.therussianissues.com/ 

Russian Observer - http://www.russianobserver.com/ 


SABA (The News Agency of Yemen) - http://www.sabanews.gov.ye 
Saudi Gazette - http://www.saudigazette.com.sa/sgazette/ 

South African Dispatch - http://www.dispatch.co.za/ 

Sydney Morning Herald - http://www.smh.com.au 
Tehran Times - http://www.tehrantimes.com/ 

Times of India - http://timesofindia.indiatimes.com/cms.dll 
UNHCR UN High Commissioner on Refugees - http://www.unhcr.ch/ 
Ummah News - http://www. umTnahn ews.com/ 

Uzbekistan Report - http://www.uzreport.com/eng/ 

Washington Post - http://www.washingtonpost.com/ 

XinHua News Service - http://www.xinhuanet.com/english/ 

Yemen Times (weekly) - http://www.yementimes.com/ 

Yomiuri Shimbim (Japan) - http://www.yomi\iri.co.jp/ 
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Outside the Scope 

1.2.2 Supported Projects 

IDW-S is an FBI enterprise program/system and utilizes data feeds from several FBI (and other 
agency) programs. 
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San Diego State University 
San Diego, CA 
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INTRODUCTION 

The Federal Bureau of Investigation’s (FBI) Investigative Data Warehouse Secret System (IDW- 
S) is an initial data warehouse, content management and data mining system that will permit FBI 
investigative, analytical, administrative and intelligence personnel to access aggregated data 
previously only available through individual applications. The IDW-S system will be authorized 
to process classified national security data up to, and including. Secret. The IDW-S system is the 
successor to the Secure Collaboration Operational Prototype Environment (SCOPE), which 
originally was named the Secure Coxmter-Terrorism Operational Prototype Environment. 



Data processed by the system will include the following data sets; 

• Approved case files from the FBI’s Automated Case Support (ACS) case 
management system; 


Outside the Scope 


Electronic versions of the Joint Intelligence Committee Investigation (JICI) 
defined archived documents; 
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• Secure Automated Messaging Network (SAMNet) message traffic; 

• IntelPlus File Rooms; and 

• Violent Gang and Terrorist Organization File (VGTOF) from the Criminal 
Justice Information Systems (CJIS) Division; 

• Defense Advanced Research Projects Agency (DARPA) Translingual 

Information Detection, Extraction and Summarization (TIDES) Open Source 
Data 


Outside the Scope 
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Outside the Scope 

1.2.2 Supported Projects 

IDW-S is an FBI enterprise program/system and utilizes data feeds from several FBI (and other 
agency) programs. 
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3.5. Data Processed 


IDW-S processes data jfrom the following data sources, with the security classification of the 
source indicated in brackets: 

• Automated Case Support (ACS) System, Electronic Case File (ECF) Subsystem 
[Secret and below] 

• IntelPlus [Secret and below] 

• Secure Automated Message Network - Secret (SAMNet-S) [Secret and below] 

• NCIC Violent Gang and Terrorist Offender File (V GTOF) [Sensitive But 
Unclassified] 


• Joint Intelligence Committee Investigation (JICI) [Secret and below] 


• DARPA/TIDES Open Source News [Unclassified] 


Outside the Scope 
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Data Sources w/Transfer Mechanism 


3.5.1 ACSECF 

IDW-S contains a subset of the ECF (Electronic Case File) subsystem of the Automated Case 
System (ACS). This subset consists of serials in those case classifications/sub classifications that 
have been officially sanctioned for inclusion in IDW-S. For each such serial, the ECF data 
includes metadata and text. IDW-S is synchronized against the ECF system once a day, a process 
which consists o f receiving and processing the previous day’s increment of ADD, MOD, and 
DELETE recordsi ~~ 


3.5.2 IntelPlus Outside the Scope 


IDW-S currently contains several IntelPlus coimter-terrorism (CT) Filerooms: 
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3.5.6 Open Source News 

The IDW-S Vl.O system contains an Open Source News library collected by the DARPA TIDES 
Program. These are primarily news source from around the world that are either in English or 
have been translated into English. Open Source N ews data is received in the form of text files 

(one per news article' The Open Source News data goes into IDW-S 

once a day. Open Source JNews data 


Outside the Scope 
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3.9.9 Indirect Connections Outside the Scope 

DDW-S connects to the following sources of data: 
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INTRODUCTION 

The Federal Bureau of Investigation’s (FBI) Investigative Data Warehouse Secret System (DDW- 
S) is an initial data warehouse, content management and data mining system that will permit FBI 
investigative, analytical, administrative and intelligence personnel to access aggregated data 
previously only available through individual applications. The IDW-S system will be authorized 
to process classified national security data up to, and including. Secret. The IDW-S system is the 
successor to the Secure Collaboration Operational Prototype Environment (SCOPE), which 
originally was named the Secure Counter-Terrorism Operational Prototype Environment. 



Data processed by the system will include the following data sets: 

• Approved case files firom the FBI’s Automated Case Support (ACS) case 
management system; 

• Electronic versions of the Joint Intelligence Committee Investigation (JICI) 
defined archived documents; 
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• Secure Automated Messaging Network (SAMNet) message traflSc; 

• IntelPIus File Rooms; and 

• Violent Gang and Terrorist Organization File (VGTOF”) from the Criminal 
Justice Information Systems (CJIS) Division; 

• Defense Advanced Research Projects Agency (DARPA) Translingual 

Information Detection, Extraction and Summarization (TIDES) Open Source 
Data 


Outside the Scope 
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• Automated Case Support (ACS) System, Electronic Case File (ECF) Subsystem 
[Secret and below] 

• IntelPlus [Secret and below] 

• Secure Automated Message Network — Secret (SAMNet-S) [Secret and below] 

• NCIC Violent Gang and Terrorist Offender File (VGTOF) [Sensitive But 
Unclassified] 

• Joint Intelligence Committee Investigation (JICI) [Secret and below] 

• DARP A/TIDES Open Source News [Unclassified] 

Data from each source is held in an identified source Library against which users can direct the 
analytical tools. 

|Oata Ingest | 

IDW-S receives data from sources by F TP over FBINet (ACS ECF, IntelPlus. and SAMNet) and 
CD-ROM (VGTOF, Open Source News)) 


• Data is received by the Ingest Subsystem 


Received data is parsed into individual serial records (in the case of ACS 
ECF), documents (IntelPlus, JICI, Open Source News), messages (SAMNet), 
or files (VGTOF). 


-16- 
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3.5. Data Processed 

IDW-S processes the following data: 

3.5.1 ACS ECF 

IDW-S contains a subset of the ECF (Electronic Case File) subsystem of the Automated Case 
System (ACS). This subset consists of serials in those case classifications/sub classificati ons that 

- 21 - . 
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have been officially sanctioned for inclusion in IDW-S. For each such serial, the ECF data 
includes metadata and text. IDW-S is synchronized against the ECF system once a day, a process 
which consists of receiving and processing the previous day’s increment of ADD, MOD, and 
DELETE records.? 


3.5.2 IntelPlus 

EDW-S currently contains several IntelPlus counter-terrorism (CT) Filerooms: 


Outside the Scope 
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3.5.3 SAMNet 

The Secure Automated Mess aging Network (SAMNet) data source consists of cable traffic 
messages received by the FBI 


I ^ 


SAMNet provides o nly ADD record types, and SAMNet data is updated in EDW-S three 
times a day. SAMNet data i j 


b2 
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I 


j 

j 


! 


3.5.4 VGTOF 


Violent Gang Terrorist Offender File (V GTOF) data is provided by the FBI National Crime 
Information Center (NCIC). VGTOF data includes two components: Data/met adata for each 
named individual/offender and potentially multiple JPEG images per individual 


3.5.5 JICI 


The Joint Intelligence Committee Investigation (JICI) data collection was created following the 
September 1 1 attacks. Paper counter-terrorism files/folders from all FBI Field Office^ 


I JICI is a static collection that is not updated 


v 


3.5.6 Open Source News 

The IDW-S VI .0 system contains an Open Source News library collected by the DARPA TIDES 
Program. These are primarily news source from around the world that are either in English or 
have been translated into English. Open Source News data is received in the form of text files 

(one per news article) anc |__ ^ |The Open Source News data goes into IDW-S 

once a day. Open Source News dat ^^^^^^ | 

jOpen Source News material is derived from the following sources: 

Addis Ababa Tribune - http://www.addistribune.com/ 

Agencia Brasilia - http://www.radiobras.gov.br/ 

Al-Ahram (Egypt, weekly English version) - http://weekly.ahram.org.eg/ 

AllAfrica.com - http://allafrica.com/ 

Arabic News - http://www.arabicnews.com/ansub/ 

Asahi Shimbun - http://www.asahi.com/english/ 

Asia Times (Hong Kong) - http://www.atimes.com/ 

Bangkok Post - http://www.bangkokpost.net/ 

Christian Science Monitor - http://www.csmonitor.com/ 
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Crescent International - http://www.muslimedia.com/mainpage.htm 
Daily Telegraph (London, England) - http://news.telegraph.co.uk/ 

Dawn (Karachi, Pakistan) - http://www.dawn.com/ 

Debka (Israel) - http://www.debka.com 

East Africa Daily Nation - www.nationaudio.com/News/DailyNation/Today/ 

Gulf News (UAE) - http://www.gulf-news.com/ 

Ha'aretz (Israel) - http://www.haaretzdaily.com/ 

IFRC International Federation of the Red Cross - http://www.ifrc.org/ 

IRIN Integrated Regional Information Network - http://www.irinnews.org/ 

Iraq Press News Agency - http://vyww.iraqpress.org/ 

Islamic Republic News Agency (Iran) - http://www.ima.com/en 
Jakarta Post - http://www.thejakartapost.com/ 

Janes - http://www.janes.com/ 

Jordan Times - http://www.jordantimes.com/ 

L'Osservatore Romano - www.vatican.va/news_services/or/or_eng/text.html 
Lagos (Nigeria) Guardian - http://www.guardiannewsngr.com/ 

Lahore (Pakistan) Nation - http://www.nation.com.pk/ 

Lebanon Daily Star (Beirut) - http://www.dailystar.com.lb/ 

Malaysian Star - http://thestar.com.my/ 

Manila Bulletin - http://www.mb.com.ph/ 

Manila Times - http://www.manilatimes.net/ 

Miami Herald - http://www.miami.com/ 

Moscow Times - http://www.themoscowtimes.com/ 

National Post and CP - http://www.canada.com/ 

New Straits Times (Kuala Lumpur) - http://www.emedia.com.my/Current_News/NST/ 

PETRA (Jordanian News Agency) - http://www.petra.gov.jo 
Pakistan Observer (Islamabad) - http://pakobserver.net/ 

Palestine Chronicle - http://palestinechronicle.com/ 

People's Daily (China) - http://english.peopledaily.com.cn/ 

Philippine Star - http://www.philstar.com/philstar/ 

Pravda - httpV/english.pravda.ru/ 

ProMed - epidemiology mailing list 

Russian Information Agency Novosti - http://en.rian.ru/ 

Russian Issues (Misc. Russian news) - http://www.therussianissues.com/ 

Russian Observer - http://www.russianobserver.com/ 

SABA (The News Agency of Yemen) - http://www.sabanews.gov.ye 
Saudi Gazette - http://www.saudigazette.com.sa/sgazette/ 

South African Dispatch - http://www.dispatch.co.za/ 

Sydney Morning Herald - http://www.smh.com.au 
Tehran Times - http://www.tehrantimes.com/ 

Times of India - http://timesofindia.indiatimes.com/cms.dll 

UNHCR UN High Commissioner on Refugees - http://www.unhcr.ch/ 
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Ummah News - http://www.ummahnews.com/ 

Uzbekistan Report - http://www.uzreportcom/eng/ 
Washington Post - http://www.washingtonpost.com/ 
XinHua News Service - http://www.xinhuanet.com/english/ 
Yemen Times (weekly) - http://www.yementimes.com/ 
Yomiuri Shimbun (Japan) - http://www.yomiuri.co.jp/ [ 

3.5.7 Summary of [Data Sources| 
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1 introduction 

The Federal Bureau of Investigation (FBI) Integrated Data Warehouse (IDW) is a data 
warehouse, content management, and data mining system that will permit FBI investigative, 
analytical, administrative, and intelligence persormel to access aggregated data previously 
available only through individual applications. The IDW-S system will be authorized to process 
classified national security data up to, and including, the Secret level. 



ata processed by the system will mclude the following data sets: Outside the scope 

• Approved case files ftom the FBI’s Automated Case Support (ACS) case 
management system; 

• Electronic versions of the Joint Intelligence Committee Investigation (JICI) 
defined archived documents; 

• Secure Automated Messagmg Network (SAMNet) message traffic; 

• IntelPlus File Rooms; 

• Violent Gang and Terrorist Organization File (VGTOF) firom the C rimina l 
Justice Information Systems (CJIS) Division; and 


• Open Source Data 
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